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Application No. ^ Appllcant(s) 

09/537,334 LOUl ETAL. 

Examiner Art Unit 

Colin M. LaRose 2623 ^ 

- The MAILING DATE of this communication appears on the cover sheet with the correspondence address - 
Period for Reply 

A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 .136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If the period for reply specified above is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timely. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 
Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1 .704(b). 

Status 

1 )S Responsive to comnnunlcation(s) filed on 26 April 2004 . 
2a)K This action is FINAL. 2b)n This action is non-final. 

3) n Since this application is in condition for allowance except for formal nnatters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 1 1 , 453 O.G. 213. 

Disposition of Claims 

4) S Claim(s) 3-17.20-24,26-28 and 30-33 is/are pending in the application. 

4a) Of the above claim(s) __ is/are withdrawn from consideration. 

5) ^ Claim(s) 8-17,26-28 and 33 is/are allowed. 

6) ^ Claim(s) 3-7,20-24,30 and 32 is/are rejected. 

7) ^ Claim(s) 31 is/are objected to. 

8) n Ciaim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) D The specification is objected to by the Examiner. 

10)0 The drawing(s) filed on is/are: a)n accepted or b)n objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1 .85(a). 

Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1.121(d). 
1 1 )□ The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-1 52. 

Priority under 35 U.S.C. § 119 

12)n Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 1 19(a)-(d) or (f). 
a)n All b)n Some * 0)0 None of: 

1 .□ Certified copies of the priority documents have been received. 

2. n Certified copies of the priority documents have been received in Application No. . 

3. n Copies of the certified copies of the priority documents have been received in this National Stage 

application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 
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Attach ment(s) 

1 ) S Notice of References Cited (PTO-892) 

2) O Notice of Draftsperson's Patent Drawing Review (PTO-948) 

3) ^ Information Disclosure Statement(s) (PTO-1449 or PTO/SB/08) 

Paper No(s)/Mail Date 12 and 14 . 



4) □ Intennew Summary (PTO-413) 

Paper No(s)/Mail Date. . 

5) □ Notice of Infomial Patent Application (PTO-152) 

6) □ Other: . 



U.S. Patent and Trademark Office 
PTOL-326 (Rev. 1-04) 
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DETAILED ACTION 
Arguments and Amendments 

1 . Applicants' amendments and arguments filed 26 April 2004, have been entered and made 
of record. 

Information Disclosure Statement 

2. The information disclosure statement filed 1 June 2004 fails to comply with 37 CFR 
1.98(a)(2), which requires a legible copy of each U.S. and foreign patent; each publication or that 
portion which caused it to be listed; and all other information or that portion which caused it to 
be listed. It has been placed in the application file, but the information referred to therein has not 
been considered. 

Response to Amendments and Arguments 

3. For claims 3, 4, 20, and 21, Applicant argues (page 14, paper 13) that Shimura does not 
disclose dividing the images into 3x3 or 4x4 blocks. In previous rejections, Examiner maintained 
that this feature was conventional in the art. U.S. Patent 6,400,853 by Shiiyama has been applied 
to show that it was conventional to divide an image into 3x3 blocks for the purposes of 
determining the similarity amongst images on the basis of the similarity between the blocks. 

4. For claims 7, 24, and 30, Applicant argues (page 12, paper 13) that Shimura does not 
teach that two or more blocks would represent a foreground area in Shimura' s images. However, 
Examiner asserts that dividing Shimura' s images into a plurality of blocks, such as image 20 
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(figure 2), would produce at least two blocks that contain the foreground object. That is, if the 
image 20 is divided into a plurality of blocks, then at least two of the blocks will contain a 
portion of the leaf 

Claim Rejections - 35 USC § 103 

5. The text of those sections of Title 35, U.S. Code not included in this action can be found 
in a prior Office action. 

6. Claims 3, 4, 7, 20, 21, 24, and 32 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over U.S. Patent 5,644,765 by Shimura et al. ("Shimura") in view of U.S. Patent 
5,805,215 by Mizoguchi and U.S. Patent 6,400,853 by Shiiyama. 

Regarding claims 3 and 20, Shimura discloses a method/computer program (figure 4) for 
detecting duplicate (i.e. substantially similar) images comprising the steps of: 

providing at least two images captured at determinable times (figure 2: image 20 and 
database of images 33 are provided; figure 2, elements 43, 12, and 32 and column 4, lines 1-9: 
additional information, such as date and time of registration, is associated with each of the 
images); 

computing an indication of image content for each image (figure 4, SI 7: image features 
relating to the content of each of the images are computed; e.g. figure 5, S30: feature extracted 
for image 20 and figure 3, S3: features extracted for database images 33); 

determining the time of capture of each of the images (figure 4, S13: additional 
information (e.g. time of registration) is determined for each image); and 




Application/Control Number: 09/537,334 Page 4 

Art Unit: 2623 

evaluating the indication of image content (figure 4, SI 8) and the time of capture (figure 
4, SI 6) to determine whether the images are duplicate images (i.e. whether the images are 
substantially similar with regards to the content of the image blocks and the time of registration); 

wherein the step of computing an indication of image content comprises: 

dividing each image into blocks (column 3, lines 33-37); 

and computing an indication of image content in each block (e.g. number of black pixels 
in each block). 



Shimura teaches inputting the images using a scanner 1 1 (or the like), figure 1, and 
inputting the additional information 12, figure 1, of input images, such as the date of capture, 
using a keyboard. 

Shimura is silent to using a photographic camera that records the time of capture. 

Mizoguchi discloses a digital camera having the ability to store additional time data along 
with the images captured by the camera (1, figure 1). The time/date of capture is encoded in 
memory 51 (figure 10) with each image provided by the camera (column 6, lines 57-65), The 
date/time information is then used as criteria for retrieving images (column 8, lines 7-28). 

It would have been obvious to one of ordinary skill in the art at the time of the invention 
to modify Shimura by Mizoguchi to utilize a Mizoguchi 's photographic camera to input images 
and the time of capture of the images, since Mizoguchi 's camera functions as an image input 
device in substantially the same fashion (i.e. to provide an electronic representation of an object) 
as Shimada's image input device 11, and Mizoguchi' s camera eliminates the need for the user to 
manually input the time of capture of the image. 
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Shimixra and Mizoguchi do not expressly disclose dividing the images into 4x4 or 3x3 
blocks. However, at the time the invention was made, dividing an image into a small number of 
blocks and processing each block was common in the art and would have been an obvious 
modification to those skilled in the art. 

As Shiiyama shows (figure 5), it is conventional to divide an image into 3x3 blocks for 
the purposes of determining the similarity between images. 

Regarding claims 4 and 21, Shiiyama discloses utilizing 3x3 image blocks for similarity 
calculations (figure 5). 

Regarding claims 7, 24, and 32, the combination of Shimura, Kim, and Shiiyama disclose 
two or more of the blocks represent a foreground area of the images (i.e. Shimura's images (e.g. 
20, figure 2), when divided into 3x3 blocks, include at least two blocks that represent a 
foreground area); and 

said computing further comprises computing an indication of image content in the 
foreground areas of each image (i.e. the images (20 and 33, figure 2) utilized by Shimura are 
essentially foreground objects; therefore, the indications of image content (column 3, lines 33- 
35: number of black pixels in each block) computed fi-om the image blocks correspond to 
foreground areas). 
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7. Claim 30 is rejected under 35 U.S.C. 103(a) as being unpatentable over Shimura in view 
of Mizoguchi. 

Regarding claim 30, Shimura discloses a method (figure 4) for detecting duplicate (i.e. 
substantially similar) images comprising the steps of: 

providing at least two images captured at determinable times from original scenes (figure 
2: image 20 and database of images 33 are provided; figure 2, elements 43, 12, and 32 and 
column 4, lines 1-9: additional information, such as date and time of registration, is associated 
with each of the images and denote determinable times of capture); 

computing an indication of image content for each image by dividing each image into 
blocks, computing an indication of image content in each block (e.g. number of black pixels in 
each block; column 3, lines 33-37), and comparing the computed indication of image content in 
each corresponding block for the two images to generate a similarity metric for each block (32, 
figure 5 and 52, figure 2: the corresponding blocks (i.e. the computed indications) of the images 
are compared to generate similarity metric); 

determining the time of original captwe of each of the images (figure 4, S13: additional 
information (e.g. time of registration) is determined for each image); and 

evaluating the similarity metric for each block and the time of original capture (figure 4, 
S16 and SI 8) to determine whether the images are duplicate images (i.e. whether the images are 
substantially similar with regards to the content of the image blocks and the time of registration); 

wherein the step of computing an indication of image content fiirther comprises assigning 
two or more of the blocks represent a foreground area of the images (i.e. Shimura's images (e.g. 
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20, figure 2), when divided into a number of blocks, include at least two blocks that represent a 
foreground area); and 

computing an indication of image content in each block and in the foreground areas of 
each image (i.e. the images (20 and 33, figure 2) utilized by Shimura are essentially foreground 
objects; therefore, the indications of image content (column 3, lines 33-35: number of black 
pixels in each block) computed from the image blocks correspond to foregroimd areas) 

Shimura teaches inputting the images using a scanner 1 1 (or the like), figure 1 . 

Shimura is silent to using a photographic camera to capture the images. 

Mizoguchi discloses a photographic camera having the ability to store additional time 
data along with the images captured by the camera (1, figure 1). The time/date of capture is 
encoded in memory 51 (figure 10) with each image provided by the camera (column 6, lines 57- 
65). The date/time information is then used as criteria for retrieving images (column 8, lines 7- 
28). 

It would have been obvious to one of ordinary skill in the art at the time of the invention 
to modify Shimura by Mizoguchi to utilize Mizoguchi' s photographic camera to input images 
and the time of capture of the images, since Mizoguchi's camera functions as an image input 
device in substantially the same fashion (i.e. to provide an electronic representation of an object) 
as Shimada's image input device 1 1, and Mizoguchi's camera eliminates the need for the user to 
manually input the time of capture of the image. 
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8. Claims 5, 6, 22, and 23 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Shimura in view of Mizoguchi, Shiiyama and U.S. Patent 6,445,818 by Kim et al. ("Kim"). 

Regarding claims 5 and 22, Shimura discloses the image feature comprises the number of 
black pixels in each block (column 3, lines 33-35). However, Shimura does not expressly 
disclose computing a histogram for each block. 

Kim discloses a system for determining the content of an image so that an accurate search 
for the image can be performed. Kim teaches that a conventional method of determining an 
indication of the image content is to compute histograms for each block in an image. Figure IC 
shows an image is first divided into blocks. Then, in figure ID, a histogram is formed for each 
block, indicating the image content. 

It would have been obvious to one of ordinary skill in the art at the time of the invention 
to modify Shimura and Mizoguchi by Kim in order to compute a histogram for each block, since 
Kim teaches histograms are conventionally used to compute indications of image content in 
image blocks, and histograms are a means to determine the number of blacks pixels in each 
block. 

Regarding claims 6 and 23, Shimura teaches comparing a block of one image (e.g. image 
20), using the extracted feature, to a corresponding block of another image (e.g. an image in 
database 33) and using the time difference between capture of the two images to deteraiine 
whether the images are duplicate images. Figure 4, SI 3: time information of the two images is 
compared; figure 4, SI 7: features of the two images are compared. Bother criteria are used to 
determine the similarity of the images. 
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With Kim's modification, the histograms of the two images are compared to determine 
similarity (colunm 2, lines 17-20). Kim does not expressly disclose the use of a histogram 
intersection metric to compare the histograms, however, using an intersection metric to the 
determine the similarity of two histograms was well-known and obvious to those skilled in the 
art. Official notice taken. 

Allowable Subject Matter 

9. Claim 3 1 is objected to as being dependent upon a rejected base claim, but would be 
allowable if rewritten in independent form including all of the limitations of the base claim and 
any intervening claims. 

Claims 33, 26, 27, and 28 are allowable because Shimura and Kim fail to disclose the 
claimed histogram intersection metric of claim 33. 
Claims 8-17 are also allowable. 

Conclusion 

10. Applicant's amendment necessitated the new ground(s) of rejection presented in this 
Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). 
AppUcant is reminded of the extension of time poUcy as set forth in 37 CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS fi'om the mailing date of this action. In the event a first reply is filed within TWO 
MONTHS of the mailing date of this final action and the advisory action is not mailed until after 
the end of the THREE-MONTH shortened statutory period, then the shortened statutory period 
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will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 
CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, 
however, will the statutory period for reply expire later than SIX MONTHS from the date of this 
final action. 

Any inquiry conceming this communication or earlier communications from the 
examiner should be directed to Colin M. LaRose whose telephone number is (703) 306-3489. 
The examiner can normally be reached Monday through Thursday from 8:00 to 5:30. The 
examiner can also be reached on alternate Fridays. 

If attempts to reach the examiner by telephone are imsuccessfiil, the examiner's 
supervisor, Amelia Au, can be reached on (703) 308-6604. The fax phone number for the 
organization where this application or proceeding is assigned is (703) 872-9306. 

Any inquiry of a general nature or relating to the status of this application or proceeding 
should be directed to the TC 2600 Customer Service Office whose telephone number is (703) 
306-0377. . f\ 
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